Win Big Results
2024-12-28
Experiment 1a
Contingencies
Low Value Options
- Fixed: 20 (Door 3)
- Risky: 0 or 40 (Door 4)
High Value Options
- Fixed: 60 (Door 1)
- Risky: 40 or 80 (Door 2)
Trials
- Six blocks of 80 trials.
- Decision Trials: (fixed low vs risky low) & (fixed high vs risky high) - 24 per block (12 low, 12 high)
- Single door trials: one door to click - 40 per block
- Catch trials: high vs low - 16 per block
EX 80-20
- Block 1: 80% extreme values (0 low, 80 high)
- Block 2: 50% extreme values.
- Block 3: 20% extreme values (0 low, 80 high)
- Block 4-6: Block 2: 50% extreme values.
EX 20-80
- Block 1: 20% extreme values (0 low, 80 high)
- Block 2: 50% extreme values.
- Block 3: 80% extreme values (0 low, 80 high)
- Block 4-6: 50% extreme values.
EX 50 (control)
- All blocks: 50% extreme values.
Participant Information
Pre-catch exclusion
N = 192
| group | gender | n | age_mean | age_sd | age_median | age_IQR |
|---|---|---|---|---|---|---|
| EX 50 | F | 38 | 19.132 | 1.711 | 18 | 1.75 |
| EX 50 | M | 25 | 20.320 | 3.198 | 19 | 3.00 |
| EX 80-20 | F | 38 | 19.447 | 3.644 | 18 | 1.75 |
| EX 80-20 | M | 28 | 19.607 | 2.347 | 19 | 2.00 |
| EX 20-80 | F | 39 | 20.179 | 4.667 | 19 | 2.00 |
| EX 20-80 | M | 24 | 19.375 | 1.952 | 19 | 2.00 |
Post-catch exclusion
N = 184
| group | gender | n | age_mean | age_sd | age_median | age_IQR |
|---|---|---|---|---|---|---|
| EX 50 | F | 36 | 19.194 | 1.737 | 18.5 | 2 |
| EX 50 | M | 22 | 20.318 | 3.272 | 19.0 | 3 |
| EX 80-20 | F | 37 | 19.378 | 3.669 | 18.0 | 1 |
| EX 80-20 | M | 27 | 19.519 | 2.343 | 19.0 | 2 |
| EX 20-80 | F | 39 | 20.179 | 4.667 | 19.0 | 2 |
| EX 20-80 | M | 23 | 19.435 | 1.973 | 19.0 | 2 |
Condition totals:
| group | n | age_mean | age_sd | age_median | age_IQR |
|---|---|---|---|---|---|
| EX 50 | 58 | 19.621 | 2.470 | 19 | 2 |
| EX 80-20 | 64 | 19.438 | 3.157 | 19 | 2 |
| EX 20-80 | 62 | 19.903 | 3.887 | 19 | 2 |
Total number of catch trial exclusions = 8.
Catch trial exclusions were based on block 5 & 6 performance.
Catch Trials
Plot - Block means with 95% CIs
Risk Preference Trials
Plot - Block means with 95% CIs
Plot - Interaction plot with 95% CIs (Block 5 & 6)
Main Effects
ANOVA Table
| Model | df | AIC | BIC | logLik | Test | L.Ratio | p.value | BF_10 | |
|---|---|---|---|---|---|---|---|---|---|
| base_mod | 1 | 4 | 226.496 | 242.128 | -109.248 | NA | NA | NA | |
| grp_mod | 2 | 6 | 228.323 | 251.772 | -108.162 | 1 vs 2 | 2.172 | 0.338 | 8.000000e-03 |
| value_mod | 3 | 7 | 176.484 | 203.840 | -81.242 | 2 vs 3 | 53.840 | 0.000 | 2.559852e+10 |
| grp_val | 4 | 9 | 179.296 | 214.469 | -80.648 | 3 vs 4 | 1.188 | 0.552 | 5.000000e-03 |
Main Effect of Group:
\(\chi^2(2) = 2.17, p = 0.338, BF_{10} < .01\)
Main Effect of Choice Value (i.e., Extreme Outcome effect):
\(\chi^2(1) = 53.84, p < .001, BF_{10} > 150\)
Interaction: Group \(\times\) Choice Value:
\(\chi^2(2) = 1.19, p = 0.552, BF_{10} < .01\)
Planned Comparisons
ANOVA Table
| Value | Std.Error | DF | t-value | p-value | sig | r_effect | |
|---|---|---|---|---|---|---|---|
| (Intercept) | 0.413 | 0.034 | 181 | 12.219 | 0.000 | TRUE | 0.672 |
| group 8020_vs_50 | 0.041 | 0.047 | 181 | 0.872 | 0.384 | FALSE | 0.065 |
| group 2080_vs_50 | -0.026 | 0.047 | 181 | -0.552 | 0.581 | FALSE | 0.041 |
| choice_value1 | 0.102 | 0.024 | 181 | 4.336 | 0.000 | TRUE | 0.307 |
| group 8020_vs_50 :choice_value1 | -0.014 | 0.032 | 181 | -0.435 | 0.664 | FALSE | 0.032 |
| group 2080_vs_50 :choice_value1 | 0.020 | 0.033 | 181 | 0.620 | 0.536 | FALSE | 0.046 |
EX 80-20 vs. EX 50:
\(b = 0.04, t(181) = 0.87, p = 0.384, r = 0.06\)
EX 20-80 vs. EX 50:
\(b = -0.03, t(181) = -0.55, p = 0.581, r = 0.04\)
EX 80-20 vs. EX 50 | Choice Value (High vs. Low):
\(b = -0.01, t(181) = -0.43, p = 0.664, r = 0.03\)
EX 20-80 vs. EX 50 | Choice Value (High vs. Low):
\(b = 0.02, t(181) = 0.62, p = 0.536, r = 0.05\)
Notes
- All plots and analyses are showing the combined effect of block 5 and 6.
- Bayes Factors are reported as Inverse Bayes Factors, this means larger values indicate greater support for the effect. Anything less than 1 should be considered as contributing no evidence.
First-Outcome Judgement
Plot - Proportions
Note:
- Proportions are taken within condition and context (high/low). E.g., All the green bars in the top left sum to 1, all the orange bars in the top middle sum to 1, and so on.
- Since non-numeric responses were permissible by the experiment’s program, data was filtered to remove all non-numeric inputs. If a participant’s inputs had a cumulative sum less than 20 or greater than 300, their results were removed. In total, 8 participants met this criteria.
2x3 Chi-Squared Tests
High Value Results
| No | Yes | |
|---|---|---|
| EX 50 | 11 | 38 |
| EX 80-20 | 13 | 44 |
| EX 20-80 | 16 | 39 |
\(\chi^2(2) = 0.81, p = 0.667, \varphi_c = 0.07\)
- Cannot rule out that recalling the extreme value is independent of group.
| No | Yes | |
|---|---|---|
| EX 50 | -0.47 | 0.47 |
| EX 80-20 | -0.44 | 0.44 |
| EX 20-80 | 0.90 | -0.90 |
| No | Yes | |
|---|---|---|
| EX 50 | 0.642 | 0.642 |
| EX 80-20 | 0.658 | 0.658 |
| EX 20-80 | 0.369 | 0.369 |
None of the three conditions differ significantly from the expected value.
Low Value Results
| No | Yes | |
|---|---|---|
| EX 50 | 12 | 37 |
| EX 80-20 | 8 | 49 |
| EX 20-80 | 7 | 48 |
\(\chi^2(2) = 3.04, p = 0.219, \varphi_c = 0.14\)
- Cannot rule out that recalling the extreme value is independent of group.
| No | Yes | |
|---|---|---|
| EX 50 | 1.73 | -1.73 |
| EX 80-20 | -0.69 | 0.69 |
| EX 20-80 | -0.99 | 0.99 |
| No | Yes | |
|---|---|---|
| EX 50 | 0.083 | 0.083 |
| EX 80-20 | 0.492 | 0.492 |
| EX 20-80 | 0.323 | 0.323 |
- None of the three conditions differ significantly from the expected value.
Post-Hoc Pairwise Fisher-Exact Tests
| value | comparison | odds_ratio | p_value | CI_low | CI_upper | p_value_adj |
|---|---|---|---|---|---|---|
| High | EX 50 : EX 80-20 | 0.9799 | 1.0000 | 0.3521 | 2.6883 | 1.0000 |
| High | EX 50 : EX 20-80 | 0.7080 | 0.5056 | 0.2603 | 1.8687 | 1.0000 |
| High | EX 80-20 : EX 20-80 | 0.7223 | 0.5201 | 0.2805 | 1.8310 | 1.0000 |
| Low | EX 50 : EX 80-20 | 1.9735 | 0.2157 | 0.6633 | 6.1822 | 1.0000 |
| Low | EX 50 : EX 20-80 | 2.2068 | 0.1357 | 0.7174 | 7.3155 | 0.8141 |
| Low | EX 80-20 : EX 20-80 | 1.1184 | 1.0000 | 0.3256 | 3.9337 | 1.0000 |
- Nothing reached significance.
Note:
In the contingency tables, ‘Yes’ indicates the extreme value was recalled, ‘No’ indicates that it was not.
Only recalled outcomes that matched the programmed values were included. i.e., if they saw the High Value Risky Door, 80 was logged as “Yes” and 40 was logged as “No.” If they saw the Low Value Risky Door, 0 was logged as “Yes”, 40 was logged as “No”. Participants who responded with any “other” values were removed from the analysis. Only Risky doors were evaluated.
In total, removing “other” responses resulted 15 participants being excluded from these analyses leaving a final N = 161.
\(\varphi_c\) is Cramér’s V, a measure of association for two nominal variables (i.e., an effect size of sorts). It can take a value between 0 and 1, with 0 indicating no association (i.e., full independence).
Post-hoc analysis on each \(\chi^2\) test was conducted by evaluating the standardized residuals from the chi-square analysis because it seemed the most intuitive approach. However, pairwise Pearson Chi-squared or Fisher Exact tests across the three conditions were also run. The
p_value_adjcolumn shows corrected p-values using the Holm-Bonferroni method.
Frequency Judgement
Plot - Means and 95% CIs
Table - Plotting Values
| value | outcome | group | n | mean | sd | median | IQR |
|---|---|---|---|---|---|---|---|
| High | +40 | EX 50 | 58 | 42.897 | 17.085 | 50 | 10.00 |
| High | +40 | EX 80-20 | 64 | 43.094 | 18.564 | 50 | 20.00 |
| High | +40 | EX 20-80 | 62 | 43.516 | 20.001 | 50 | 30.00 |
| High | +80 | EX 50 | 58 | 49.966 | 16.012 | 50 | 8.75 |
| High | +80 | EX 80-20 | 64 | 52.672 | 20.788 | 50 | 15.25 |
| High | +80 | EX 20-80 | 62 | 50.210 | 17.479 | 50 | 20.00 |
| Low | 0 | EX 50 | 58 | 62.017 | 17.046 | 60 | 28.75 |
| Low | 0 | EX 80-20 | 64 | 60.219 | 23.109 | 60 | 26.25 |
| Low | 0 | EX 20-80 | 62 | 60.968 | 18.313 | 60 | 28.75 |
| Low | +40 | EX 50 | 58 | 32.466 | 17.017 | 30 | 30.00 |
| Low | +40 | EX 80-20 | 64 | 33.875 | 20.408 | 35 | 30.00 |
| Low | +40 | EX 20-80 | 62 | 32.410 | 19.266 | 33 | 30.00 |
Experiment 1b
Contingencies
Low Value Options
- Fixed: 20 (Door 3)
- Risky: 0 or 40 (Door 4)
High Value Options
- Fixed: 60 (Door 1)
- Risky: 40 or 80 (Door 2)
Trials
- Six blocks of 80 trials.
- Decision Trials: (fixed low vs risky low) & (fixed high vs risky high) - 24 per block (12 low, 12 high)
- Single door trials: one door to click - 40 per block
- Catch trials: high vs low - 16 per block
BEST 80-20
- Block 1: 80% chance of best outcome on risky options.
- Block 2: 50% of best outcome on risky options.
- Block 3: 20% chance of best outcome on risky options.
- Block 4-6: 50% of best outcome on risky options.
BEST 20-80
- Block 1: 20% chance of best outcome on risky options.
- Block 2: 50% of best outcome on risky options.
- Block 3: 80% chance of best outcome on risky options.
- Block 4-6: 50% of best outcome on risky options.
BEST 50 (control)
- All blocks: 50% of best outcome on risky options.
Participant Information
Pre-catch exclusion
N = 190
| group | gender | n | age_mean | age_sd | age_median | age_IQR |
|---|---|---|---|---|---|---|
| BEST 50 | F | 50 | 19.220 | 3.699 | 18.5 | 1 |
| BEST 50 | M | 16 | 19.375 | 1.544 | 19.0 | 2 |
| BEST 80-20 | F | 48 | 19.500 | 3.156 | 18.0 | 1 |
| BEST 80-20 | M | 13 | 19.231 | 1.787 | 19.0 | 1 |
| BEST 20-80 | F | 38 | 18.842 | 1.636 | 18.0 | 1 |
| BEST 20-80 | M | 25 | 19.840 | 1.748 | 19.0 | 2 |
Post-catch exclusion
N = 183
| group | gender | n | age_mean | age_sd | age_median | age_IQR |
|---|---|---|---|---|---|---|
| BEST 50 | F | 46 | 19.326 | 3.842 | 19 | 1 |
| BEST 50 | M | 15 | 19.467 | 1.552 | 19 | 2 |
| BEST 80-20 | F | 48 | 19.500 | 3.156 | 18 | 1 |
| BEST 80-20 | M | 13 | 19.231 | 1.787 | 19 | 1 |
| BEST 20-80 | F | 36 | 18.889 | 1.670 | 18 | 1 |
| BEST 20-80 | M | 25 | 19.840 | 1.748 | 19 | 2 |
Condition totals:
| group | n | age_mean | age_sd | age_median | age_IQR |
|---|---|---|---|---|---|
| BEST 50 | 61 | 19.361 | 3.411 | 19 | 1 |
| BEST 80-20 | 61 | 19.443 | 2.907 | 18 | 1 |
| BEST 20-80 | 61 | 19.279 | 1.752 | 19 | 2 |
Total number of catch trial exclusions = 7.
Catch trial exclusions were based on block 5 and 6 performance.
Catch Trials
Plot - Block means with 95% CIs
Risk Preference Trials
Plot - Block means with 95% CIs
Plot - Interaction plot with 95% CIs (Block 5 & 6)
Main Effects
ANOVA Table
| Model | df | AIC | BIC | logLik | Test | L.Ratio | p.value | BF_10 | |
|---|---|---|---|---|---|---|---|---|---|
| base_mod | 1 | 4 | 247.353 | 262.964 | -119.677 | NA | NA | NA | |
| grp_mod | 2 | 6 | 249.708 | 273.124 | -118.854 | 1 vs 2 | 1.645 | 0.439 | 6.000000e-03 |
| value_mod | 3 | 7 | 157.116 | 184.435 | -71.558 | 2 vs 3 | 94.592 | 0.000 | 1.813658e+19 |
| grp_val | 4 | 9 | 153.345 | 188.469 | -67.673 | 3 vs 4 | 7.771 | 0.021 | 1.330000e-01 |
Main Effect of Group:
\(\chi^2(2) = 1.65, p = 0.439, BF_{10} < .01\)
Main Effect of Choice Value (i.e., Extreme Outcome effect):
\(\chi^2(1) = 94.59, p < .001, BF_{10} > 150\)
Interaction: Group \(\times\) Choice Value:
\(\chi^2(2) = 7.77, p = 0.021, BF_{10} = 0.133\)
- Note: while the p-value is technically significant, the Bayes Factor implies the interaction contributes nothing worthwhile.
Planned Comparisons
ANOVA Table
| Value | Std.Error | DF | t-value | p-value | sig | r_effect | |
|---|---|---|---|---|---|---|---|
| (Intercept) | 0.383 | 0.033 | 180 | 11.747 | 0.000 | TRUE | 0.659 |
| group 8020_vs_50 | 0.024 | 0.046 | 180 | 0.511 | 0.610 | FALSE | 0.038 |
| group 2080_vs_50 | 0.058 | 0.046 | 180 | 1.267 | 0.207 | FALSE | 0.094 |
| choice_value1 | 0.159 | 0.022 | 180 | 7.329 | 0.000 | TRUE | 0.479 |
| group 8020_vs_50 :choice_value1 | 0.013 | 0.031 | 180 | 0.435 | 0.664 | FALSE | 0.032 |
| group 2080_vs_50 :choice_value1 | -0.067 | 0.031 | 180 | -2.173 | 0.031 | TRUE | 0.160 |
BEST 80-20 vs. BEST 50:
\(b = 0.02, t(180) = 0.51, p = 0.61, r = 0.04\)
BEST 20-80 vs. BEST 50:
\(b = 0.06, t(180) = 1.27, p = 0.207, r = 0.09\)
BEST 80-20 vs. BEST 50 | Choice Value (High vs. Low):
\(b = 0.01, t(180) = 0.43, p = 0.664, r = 0.03\)
BEST 20-80 vs. BEST 50 | Choice Value (High vs. Low):
\(b = -0.07, t(180) = -2.17, p = 0.031, r = 0.16\)
Notes
- All plots and analyses are showing the combined effect of block 5 and 6.
- Bayes Factors are reported as Inverse Bayes Factors, this means larger values indicate greater support for the effect. Anything less than 1 should be considered as contributing no evidence.
First-Outcome Judgement
Plot - Proportions
Note:
- Proportions are taken within condition and context (high/low). E.g., All the green bars in the top left sum to 1, all the orange bars in the top middle sum to 1, and so on.
- Since non-numeric responses were permissible by the experiment’s program, data was filtered to remove all non-numeric inputs. If a participant’s inputs had a cumulative sum less than 20 or greater than 300, their results were removed. In total, 6 participants met this criteria.
2x3 Chi-Squared Tests
High Value Results
| No | Yes | |
|---|---|---|
| BEST 50 | 15 | 43 |
| BEST 80-20 | 10 | 47 |
| BEST 20-80 | 23 | 30 |
\(\chi^2(2) = 9.31, p = 0.01, \varphi_c = 0.24\)
- Recalling the extreme value is dependent on condition.
| No | Yes | |
|---|---|---|
| BEST 50 | -0.56 | 0.56 |
| BEST 80-20 | -2.27 | 2.27 |
| BEST 20-80 | 2.89 | -2.89 |
| No | Yes | |
|---|---|---|
| BEST 50 | 0.572 | 0.572 |
| BEST 80-20 | 0.023 | 0.023 |
| BEST 20-80 | 0.004 | 0.004 |
The BEST 80-20 and BEST 20-80 conditions both differ significantly from the expected value. The BEST 80-20 condition tends to recall the extreme value, whereas BEST 20-80 does not.
BEST 80-20: \(z = 2.27, p = 0.023\)
BEST 20-80: \(z = -2.89, p = 0.004\)
Low Value Results
| No | Yes | |
|---|---|---|
| BEST 50 | 5 | 53 |
| BEST 80-20 | 12 | 45 |
| BEST 20-80 | 7 | 46 |
\(\chi^2(2) = 3.7, p = 0.157, \varphi_c = 0.15\)
- Cannot rule out that recalling the extreme value is independent of group.
| No | Yes | |
|---|---|---|
| BEST 50 | -1.52 | 1.52 |
| BEST 80-20 | 1.80 | -1.80 |
| BEST 20-80 | -0.27 | 0.27 |
| No | Yes | |
|---|---|---|
| BEST 50 | 0.128 | 0.128 |
| BEST 80-20 | 0.072 | 0.072 |
| BEST 20-80 | 0.786 | 0.786 |
- None of the three conditions differ significantly from the expected value.
Post-Hoc Pairwise Fisher-Exact Tests
| value | comparison | odds_ratio | p_value | CI_low | CI_upper | p_value_adj |
|---|---|---|---|---|---|---|
| High | BEST 50 : BEST 80-20 | 1.6325 | 0.3667 | 0.6105 | 4.5340 | 0.9612 |
| High | BEST 50 : BEST 20-80 | 0.4583 | 0.0713 | 0.1883 | 1.0884 | 0.3526 |
| High | BEST 80-20 : BEST 20-80 | 0.2809 | 0.0037 | 0.1037 | 0.7154 | 0.0223 |
| Low | BEST 50 : BEST 80-20 | 0.3569 | 0.0705 | 0.0914 | 1.1908 | 0.3526 |
| Low | BEST 50 : BEST 20-80 | 0.6226 | 0.5456 | 0.1453 | 2.4571 | 0.9612 |
| Low | BEST 80-20 : BEST 20-80 | 1.7435 | 0.3204 | 0.5713 | 5.7317 | 0.9612 |
- Recalling the extreme value was found to be dependent on condition (\(p = 0.022\)) only when high value options in the BEST 80-20 and BEST 20-80 conditions are considered (Row 3 in the table). Everything else fails to reach significance.
Note:
In the contingency tables, ‘Yes’ indicates the extreme value was recalled, ‘No’ indicates that it was not.
Only recalled outcomes that matched the programmed values were included. i.e., if they saw the High Value Risky Door, 80 was logged as “Yes” and 40 was logged as “No.” If they saw the Low Value Risky Door, 0 was logged as “Yes”, 40 was logged as “No”. Participants who responded with any “other” values were removed from the analysis. Only Risky doors were evaluated.
In total, removing “other” responses resulted 9 participants being excluded from these analyses leaving a final N = 168.
\(\varphi_c\) is Cramér’s V, a measure of association for two nominal variables (i.e., an effect size of sorts). It can take a value between 0 and 1, with 0 indicating no association (i.e., full independence).
Post-hoc analysis on each \(\chi^2\) test was conducted by evaluating the standardized residuals from the chi-square analysis because it seemed the most intuitive approach. However, pairwise Pearson Chi-squared or Fisher Exact tests across the three conditions were also run. The
p_value_adjcolumn shows corrected p-values using the Holm-Bonferroni method.
Frequency Judgement
Plot - Means and 95% CIs
Table - Plotting Values
| value | outcome | group | n | mean | sd | median | IQR |
|---|---|---|---|---|---|---|---|
| High | +40 | BEST 50 | 61 | 39.344 | 20.886 | 40 | 30 |
| High | +40 | BEST 80-20 | 61 | 45.000 | 15.028 | 50 | 10 |
| High | +40 | BEST 20-80 | 61 | 49.770 | 21.064 | 50 | 30 |
| High | +80 | BEST 50 | 61 | 51.934 | 17.905 | 50 | 20 |
| High | +80 | BEST 80-20 | 61 | 50.492 | 12.573 | 50 | 20 |
| High | +80 | BEST 20-80 | 61 | 43.705 | 18.557 | 50 | 20 |
| Low | 0 | BEST 50 | 61 | 67.623 | 16.447 | 70 | 30 |
| Low | 0 | BEST 80-20 | 61 | 63.770 | 18.988 | 60 | 30 |
| Low | 0 | BEST 20-80 | 61 | 61.967 | 19.880 | 60 | 30 |
| Low | +40 | BEST 50 | 61 | 25.000 | 17.442 | 20 | 30 |
| Low | +40 | BEST 80-20 | 61 | 30.881 | 18.111 | 30 | 30 |
| Low | +40 | BEST 20-80 | 61 | 32.377 | 18.203 | 30 | 30 |
R Citations
R version 4.4.1 (2024-06-14 ucrt)
- ggh4x (0.2.8) (van den Brand, 2024)
- Hmisc (5.1.3) (Harrell Jr, 2024)
- nlme (3.1.164) (Pinheiro et al., 2023)
- RColorBrewer (1.1.3) (Neuwirth, 2022)
- rcompanion (2.4.36) (Mangiafico, 2024)
- tidyverse (2.0.0) (Wickham et al., 2019)